Block floating point numeric format allows for scaling dynamic range and precision independently. By lowering precision, system performance of processors, such as of hardware accelerators, can be increased. However, lowered precision can affect system accuracy. For example, block floating point numeric format can be used in neural networks that may be implemented in many application domains for tasks such as computer vision, robotics, speech recognition, medical image processing, computer games, augmented reality, virtual reality and others. While the lowered precision can increase the performance in different functions of the neural network, including the speed at which classification and regression tasks for object recognition, lip reading, speech recognition, detecting anomalous transactions, text prediction, and many others can be performed, neural network accuracy can be adversely affected.